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(57) Abstract 

A data warehouse system and method. The data warehouse includes a memory and a processor. The memory includes a database 
having a plurality of data entries. The processor includes a cache and an override engine, wherein the cache includes a subset of the 
plurality of data entries and wherein the override engine extracts data from the cache for viewing by a user, modifies the data in response 
to one or more user commands and saves the user commands to a file for later application to the database. 
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SYSTEM AND METHOD FOR UPDATING A MULTI-DIMENSIONAL DATABASE 



Rarkgroun d of the Invention 

Field of the Invention 

The present invention relates generally to database management, and more 
particularly to a system and method for manipulating facts in multi-dimensional databases. 

Background Information 

Business decision-makers operating in today's rapidly changing business enviroiunent 
need answers to a host of questions that directly impact their ability to compete in the 
marketplace. To manage and use information competitively, many companies are 
establishing decision support systems built aroxmd a data warehouse. A data warehouse 
stores a company's operational and historical data in an integrated relational database for 
decision support applications, business data access and reporting. Decision support systems 
access such databases to analyze and summarize corporate performance. 

Data warehouses employ relational database management systems that use a language 
such as SQL to retrieve rows and coliunns of numeric data. The systems may also permit 
access to textual files such as doctmients. Data may be accessed directly via user-generated 
SQL commands, or indirectly, via an interface which generates the desired SQL commands. 
AppUcations such as Business Objects fi-om Business Objects S.A., France, 
(http://www.businessobjectsxom). Forest and Trees from Platinum Technology Inc., 
Oakbrook Terrace, IL, (http://wvvw.platinum.com), and Pilot's Lightship from Pilot Software 
Inc., Cambridge, MA, (http://www-pilotsw.com) are typical of off-the-shelf apphcations 
which use browse windows vmder the control of end-users to generate the SQL code needed 
to analyze the data in the data warehouse. 

This approach, however, runs into significant perfonnance problems associated with 
PC and network limitations. Queries generated by inexperienced users can dominate and 
crash the database, or cause excessive network congestion. In addition, there is no 
mechanism for shifting large processes so they executie during off-peak hours. 

OLAP (OnLine Analytic Processing) technology, also called multidimensional 
analysis can also be used to access the data warehouse relational database. Multidimensional 
analysis systems have been available for over 15 years, first on mainfi-ames and then on 
client/servers. Under multidimensional analysis, data is divided into the dimensions and facts 
needed to manage the business. Dimensions for marketing applications may include 
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p^n^U. di«m»«ionchanndsandti»epcriods. Th. dimensions arc „sed.o 
^rf^specificpointsinadattbasc >«ha. to. point rep«.en« is called a fac,. As 
""""^ 1-1- .niiw. nimensionsarefiirtherdescnbedby 

exanvles. uniB sold, revenue, and pnce are all facB. Dunensions ar 
attributes, such as size, flavor, location or fiscal year. Atm-bntes also descnbe hte^htes 
5 „i.bi„adin,ension,cvenoverlappingandi«o.sis«ntUera,chics. 'n'-^"-'"- 

detem.ne.heverUca.re.aUonsbipswi.hinadin,ension. For exan^le, ma Period dnnens,on. 
asUmdardhie^hyisyear-><,uar.er.>mon.h->week->d.y.Bydefi.^g*^ 
bierarcMes. it becomes possible for OLAP applications to amomaUcally shtft " 
or down a hierarchy. This is commonly rcfcred .o as -drilling' wthm thts apphcatton space, 
,0 anexampleof,hiswouldbeshiftinganan„ualrepon's.o«.U997da.adown«>v,ew.he 

individual quarter's numbers. . , . ^ 

Multidimensional analysis aUows users to select, summarize, calculate, format and 
.^r. by dimensions and by attributes within dimensions. It can be used to support vrbially 
Xame-series decision support appUcadon including reporting, analysis, forecasting and 

,5 """8^":^^^^^ _^.^^_^^p„„^^,„^^r,analysisbased„otonlyon 

historical databu. also onprojections for figure activities. For insta„ce,marketingm=^^^ 
projectsalesforthenextthreemonU^. These figures may ti,e« be introduce, mto a m«W 
J to time manufacuuing output over that period of time. To date, such analyse h^ b.«. 

System and meU>od of exacting ^d modifying infom«tion fiom an extsting database wh.ch 
can be appUed to a relational database in order .0 fice the organization from the space 
Bmitations of multidimensional databases. In addition, what is needed is a system a»l 
meUKHicapableofcreatingreportsnot only basedon existing information but also on 

25 projections of future activities. 

c;^,T,iir.ary nf the Tnvfintion 
mpresen. inv«,tionisada,a warehouse system andmetiiod. The data warehouse 

includes a memory and a pmcessor. The menK>,y includes a database having a plurahty of 
dau entries. The processor includes a cache a.»i an override engine, wherem the cad. 
30 includesasubse.ofthepluralityofdataentriesandwherein,heove,rideengmeex.rac.sda.a 

fl„mti.e cache for viewingbyauser.modifiesthedatainresponse to one ormore user 
eommands and saveslheuser commands toafile for Uter application to the database. 
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According to another aspect of the present invention, ia data warehouse is described. 
The data warehouse includes a plurality of workstations connected to a memory by a server. 
The memory includes a database having a plurality of data entries. The server includes a 
cache and an override engine, wherein the server operates in response to user commands to 

5 store a subset of the plurality of data entries in the cache, wherein the override engine extracts 
data from the cache and sends the data extracted from the cache to the client workstation for 
viewing by a user, modifies the data in response to one or more of the user commands, saves 
the user commands to a file and operates in response to a conunit conamand to modifies the 
database based on the user commands stored in the file. 

1 0 According to yet another aspect of the present invention, a method of reporting data 

from a data warehouse is described in which the steps are providing a server and a memory 
device, storing a database in the memory device, wherein the database includes a plurality of 
data entries, extracting a subset of data entries from the database, storing the subset of data 
entries on the server, modifying the data entries stored on the server in response to user 

1 5 commands, reading data from the modified data entries stored on the server and displaying 
the data to the user. 

According to yet another aspect of the present invention, a method of forecasting 
based on data in a data warehouse is described in which the steps are providing a server and a 
memory device, storing a database in the memory device, wherein the database includes a 
20 plurality of data entries, extracting a subset of data entries from the database, storing the 

subset of data entries on the server, modifying the data entries stored on the server in response 
to user commands, storing the user conmiands, reading data from the modified data entries 
stored on the server, displaying the data to the user and modifying the database based on the 
stored user commands. 

25 According to yet another aspect of the present invention, a method of increasing the 

speed in which changes to a relational database are reflected back to the user is described in 
which the steps are extracting a subset of data from the database, wherein the step of 
extracting includes the steps of displaying a representation of the subset of data to the user 
and storing the subset of data in a cache, receiving a data modification command, storing the 

30 data modification command in a file and applying the data modification conmiand against the 
subset of data stored in the cache, wherein the step of applying the data modification 
command includes the step of modifying the subset of data to reflect application of the data 
modification command. 
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to fte drawings, where like numerals refer to like eomponents Aroughou. fte several 
Figure 1 shows a data wardwuse decision support system acconiing to the present 



invention; . . 

Figure2 shows amore detailed implementation of the data warehouse decis^^^ 

support system of Figure 1 ; ♦i. 
Figure 3 illustrates a star schema implementation of a data warehouse accordmg to the 

present invention; 

Figure 4 is a more detailed description of a data warehouse according to the present 
invention; 

Figure 5 shows an alternate embodiment of data warehouse and decision support 
system according to the present invention; and 

Figures 6a and 6b illustrate distribution of adjusted data across levels in an unlocked 

> and a locked system, respectively. 

p^c-^ nr*^"" "< ^ thft Prfff^'^ Fmbodiments 
In the following detailed description of the preferred embodiments, reference is made 
to the accompanying drawings which formapart hereof, and in which is shownby way of 

illustration specific embodiments in which the invention may be practiced. It is to be 
understood that other embodiments may be utilized and structural changes may be made 
without departing from the scope of the present invention. 

Figure 1 illustrates a computer system 10 having an enhanced capacity to extract and 
modify data stored inadatabase. Computer system lOincludesamemorylZconnectedtoa 

pn,cessor 14. Processor 14 includes a cache 16, a reporting tool 19 and an override engine 
18. Memory 12 isused to storeadatabase20. Database 20 includes a plurality of data 

entries 26, 

In one embodiment, processor 14 also includes an instruction application pmcess 34 
for permanently applying the modification made withm override engine 18to data entries 26 

stored in database 20. In one such embodiment, database 20 is a data warehouse and ovemde 
engine 18 is implemented as a multidimensional data entry/edit software engmeprocess 
which supports the creation and adjustment of data points in the data warehouse. 
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In one embodiment, as is shown in Figure 2, cache 16 is implemented as a persistent 
cache which stores a subset of the plurality of data entries 26. In one such embodiment, 
override engine 1 8 extracts data from the persistent cache for viewing by a user, modifies the 
data in response to one or more user commands and saves the user commands to an 
instruction file 30 for later application to database 20. 

A user sends commands to processor 14 over command interface 22 and receives 
reports based on data extracted from database 20 through report interface 24. In one 
embodiment, as is shown in Figure 2, command interface 22 and report interface 24 are 
implemented as a single graphical user interface (GUI) 32. 

User action within system 10 is basically an adjustment process of data representing 
ftiture periods in time. In one embodiment, this data will have been pregenerated by other 
systems and stored in the database 20. An example of this would be a statistically created 
forecast of fixture volume. A user can then "override" or adjust the values (i.e., facts) that 
they consider to be inaccurate. The combination of the adjustment capabiUty with the OLAP 
ability to present the data in virtually any level of granularity allows business professionals to 
review data within a familiar business context and use their knowledge of the business to 
refine the data warehouse values. 

As with a standard read only OLAP reporting system. System 10 allows end users and 
administrators to define reports to organize and present data. These reports are created by 
selecting the desired dimensional identifiers as well as the facts that contain the data needed 
to support the decision or planning process. As an example, a report may contain facts such 
as 'Annual Plan,' 'Statistical Forecast,' and 'Last Years Actuals' with dimensions 'Central 
Region,' 'Com Syrup,' and 'May 98, June 98, and July 98.' 

All the data the user sees on the report is stored in a work file that was created for that 
adjustment cycle. The process administrator creates one work file for each adjustment cycle. 
For example, in a monthly planning cycle there would be one woik file for the May 1996 
cycle, one for the June 1996 cycle, and so on. 

When the user adjusts a fact value, the information is saved in a report file, not the 
work file. In the embodiment, when the final adjustment of the fact is completed, the process 
administrator updates the database with the new forecast data. 

As an example, the following chart summarizes a typical adjustment cycle. In this 
case, a cycle used to create a consensus tactical forecast is shown. The chart identifies the 
reqmred input, the steps, and the results. 

<WO @957658A1 I > 
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Override Process: Tactical Forecast Cycle 


Input to the Cycle 


Steps of the Cycle 


Results of the Cycle 


Statistical forecast data. 


1. Users create reports using 


Preliminary reports. 


tactical work file. 


iaiters and dimensions. 




tactical fact. 


They work only with the 






assigned proaucts/ 






markets/penods. 






2. Users adjust last month's 


1 acticai lorecasi 




tactical fact to create this 


reports lor omers lo 




month's tactical forecast. 


T*^\rif»\x/ wrn*lf" "WltH and 




They save the forecast 


change. 




amounts, and distribute 






the forecasts for review. 






3. Users make final changes 


Final version of tactical 




and save the tactical 


forecasts. 




forecasts. 




Report files, tactical 


4. Forecasting administrator 


Management work file. 


work file. 


completes the cycle. 


management fact. 



As an additional example in which the override function is used in a forecasting 
application, the following chart sununarizes a subjective management forecasting cycle. It 
1 0 identifies the required input, the steps, and the results. 
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Override Process: Management Forecast Cycle 


Input to the Cycle 


Steps f the Cycle 


Results of the Cycle 


Management work file. 


1. Users create reports using 


Preliminary reports. 


management fact. 


filters and dimensions. 






They work only with the 






assigned products/ 






markets/periods. 






2. Users adjust last month's 


Reports for others to 




management fact to 


review, work with, and 




create this month's 


change. 




management forecast. 






They save forecast 






amounts, and distribute 






the forecast for review. 






3. Users make final changes 


Final version of 




and save the management 


management forecasts. 




forecast. 




Report files. 


4. Forecasting administrator 


Final business fact. 


management work file. 


completes the cycle. 





At the end of each cycle, the process administrator prepares the data for the next 
cycle. This involves saving the data to the database, preparing the work file for the next 
cycle, and setting up the single adjustable fact for that cycle. The adjustable fact is the fact 
10 that can be changed or overridden. 

For example, at the end of the tactical forecast cycle, the Forecasting administrator 
updates the database with the consensus tactical forecast and sets up the next plaiming cycle, 
in this case a consensus management forecast. The following chart lists the steps. 
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Input to the Cycle 


Steps taken by the Forecasting 
administrator 


Results of the Cycle 


Work file for the cycle 
(tactical work file, and 
management work file) 


1. Appends all report files 

fix>m all users. 

2. Commits the data to the 

Forecasting database, 

3. Sets the fact to be 
adjusted in the next 
forecasting cycle. . 


Integration of forecast 
rules. 

Updated Forecasting 
database. 

A new adjustable fact 
for the next cycle. 



10 



A forecast override application sueh as system 10 allows use« to apply <,ualitaUv. 
ra^ (i.e.. business l=no»leage) to me forecasting process. Users can change values 
generated by purely statistical methods to reflect changing business condtuons. 

taremb«liment.sys,em.OmaintainsamasteracUvityfile. The master achv«y 
Sleconsis,softhefullsetofdin.ensioninformationuserinformation.a„dsecun.y 

information. I. also contains detailed infonnation about al, of the facts that were ch^ by 
thcptocess administrator to support thepl^uting or adiustnrent process, fttsstoredmonly 

"^dividual users have their own corfiguration files and the approphate fact data flea. 
Whentheoverrideapp.icationisstar.ea.«.emasterda«tfi.cisreadandth.«-to — 

class is built inrnemory for the user, llteda.astrucn.rehaspoin.ers to an of*ed^^ 
master structurepoints .o. A system end user can view and modify any ^ that he ^ 
permission to see. The data that they are allowed to see can be configured on . user by user 

basis to ensure that they see data they are authorized to access and in a usable context. 

Alock me is coated while the work file isbeingprocessedbyauser to ^tsurea^mtc 

20 usage of the data in the file. 

"^^h, one embodiment, reports are created from a template. The template is like a 
htaeprim fbr the repor. Each time a report is created, system 10 saves the bluepnnt. 

h. one embodhn^tt. report templates are shar«i among users. Each user can *en 
25 createth^rownversionofthereport For example, they can run a new report, mod,^ the 
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report template, or "driir to create a different version of the report. By drilling, they are able 
to review and adjust the data anew, with more discreet levels of detail. As an example, 
instead of simply adjustmg the fact ^Planned Budget* up by 25% at the annual level, the end 
user may 'drill down' to the monthly level and review and adjust specific months values. 
5 All reports contain one or more columns and one or more rows. Each report can 

consist of multiple sections. If a report contains multiple sections, each section contains the 
same type of row and colimin information. 

The section, column, and row names in a report reflect the dimensions of database 20. 
These dimensions give meaning to the values in database 20. It is, therefore, important to 
1 0 imderstand how the dimensions work together. 

Every data value in database 20 is defined by one or more dimensions. To fiilly 
describe each value, a report must contain at least one element from each of the available 
dimensions. 

Period dimensions are time intervals used for identifying and consolidating the data, 

15 such as weekly, monthly, quarterly, and yearly intervals. Non-period dimensions describe 
other aspects of your data. Non-period dimensions may include, for example, Geo-political, 
Product, and Business jOrg, 

Each dimension can consist of hierarchical levels. For example, product may be a 
dimension. Brand and SKU are associated product dimension levels. The levels represent 

20 differing degrees of detail and the paths used when you drill. 

In one embodiment, database 20 is a data warehouse stored in one or more work files. 
Each work file can contain an unlimited number of data points. (That is, the work file size is 
limited only by the amount of physical storage.) In one such embodiment database 20 is 
represented as a multidimensional database. Override engine 1 8 can modify and view data 

25 points at any combination of levels within the multidimensional database and, as values are 
changed, the new values are allocated to all levels of the dimension hierarchies. This ensures 
that summary levels of the data still total correctly. User instructions to the override engine 
18 are not applied to the data points. Instead the instructions are stored in instruction file 30. 
Since a single instruction may change tens of thousands of rows, just storing the instructions 

30 is much faster and less expensive than storing all the changed values. Once editing is 

complete the edits can be applied to the work file by executing instruction application process 
34. In this way, large scale data warehouse updates can be performed offline while the user is 
working on other tasks. 
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By storing the instructions rather than automatically applying them, computer system 
10 can present the user with the results of the desired changes more quickly than in previous 
data warehouse analysis products. In addition, hy using one or more work files rather thana 

predefined multidimensional database (MDDB) product, system 10 can handle data 
warehouses which are larger in size than cunrent database products. 

to one embodiment, override engine 18 extracts the subset of data entries 26 fiom 
database 20 and stores the extracted data into cache 1 6. In one such embodiment, ovemde 
engine 18 employs the same mechanism used by reporting tool 19 to extract data fiom 
database 20. For instance, in one such embodiment, both override engine 18 and reportmg 
tool 1 9 detemune the appropriate levels of data to extract from database 20 by querying an 
OLAP object running on a standard request broker. Similarly, both override engine 1 8 and 
reporting tool 19 look to a Metadata table such as Metadata table 36 to identify fact tables, 
determine the levels they need in each fact table, and to extract the data from each reqmred 
fact table. Override engine 18 then takes the report generated and pushes it into cache 16. 
From that point on. override engine 18 receives a command, executes the command, and 

presents the data to the user. In addition, ovenride engine 18 stores the command m 
instruction file 30 so that if one wants to remn that report later,all that has to happen is that 

the commands are reapplied to the known starting point. . 

As noted above, override engine 1 8 provides the user or users with a mechanism for 
quickly adjusting data stored in the data warehouse during a subjective fact adjustment 
session. In one embodiment, data stored in the data warehouse can be in any of three states 
during such an adjustment cycle. Data that came out of database 20 is "mitouched." 
Commands to alter the data are stored to instruction file 34 and are used to change the data 
being displayed to the user. The commands do not, however, change the data in either cache 
16 or database 20. 

Data that's been stored back to the data warehouse (i.e., stored in database 20) is 
"adjusted" or 'Svritten" data. Data in database 20 is modified using the mstructions stored m 
instruction file 34 via a "commit" command. In one such embodiment, data stored m 
persistent cache 16 is invalidated during execution of the "commit" command. 

Data which has been adjusted in cache 1 6 but which has not yet been wntten to 
database 20 is in that more nebulous state in the middle where it is adjusted but not 
committed. And that's the situation where cache 16 does not match what's m the data 
warehouse anymore. In one such embodiment, the data gets to that state by executang a 
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"release" command in engine 18. The •Release" command causes override engine 18 to apply 
the commands not only to the data presented to the screen but also to the data within cache 16 
itself. In one such embodiment, commands must be "released" before the resulting changes 
are "committed*' to database 20. Once a set of commands are "released", however, the 
original data is gone. 

In another embodiment, override engine 18 maintains a stack of old states and the user 
can scroll back through a list of releases to recover an older state of cache 16. 

In one embodiment, system 10 only extracts data from database 20 at the start of the 
forecasting process. In such an embodiment, system 10 only extracts data from database 20 
at the start of the forecasting process. In such an embodiment. System 10 applies a three-step 
process: work file generation, interactive overriding, and database storage. Work file 
gCTcration is accomplished using the extract phase. Extract reads the base statistical forecast 
data from database 20 and stores the data in a Work File (cache 16) in the UNIX file system 
of processor 14. 

The remaining phases (override, release, and commit) use the data in the Work file to 
generate adjusted fact data. System 10 allows the user to interactively change individual 
product or market values to reflect changing business conditions. Each adjustment is 
allocated up and down the dimension hierarchy (aggregation levels). Allocations are 
performed by using the dimension's drill hierarchy to identify lower level components of the 
data point adjusted. These lower level components are then given the new value that 
maintains their relative contribution to adjusted value. Users may also lock or unlock 
individual values so that the values do not change during the adjustment process (directly or 
indirectly). 

Once the adjustor has completed all work for an override, the adjustment is checked- 
in to the work file. The check-in process ensures that two adjustors do not adjust the same 
values and, in one embodiment, it allows a process administrator to review the work before 
commit. 

The final phase of the override process is the database storage phase, or commit. 
Once the user or administrator is satisfied with the generated overrides, the overrides must be 
committed to the database. This is accomplished in the storage phase. 

As noted above, extraction is the first phase of the override process. It is the 
generation of the work file. Extraction builds a snapshot of database 20 and stores the 
snapshot in the Work File in the UNIX file system of processor 14. Extraction is generally 
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a.™ «»ebe«»sei. is a«n.«— gp»c«s and you gen«aUy do no, wan, or ne«. a 
nsertos^U^tbappen. U, one enJ»d™«. U« «»a«ion process is perfom.ed by an agen, 
..ec„tingwi.hinreportingU»l 19. Tl* ag«« Wically be set »p to perform fte 
extraction process after database 20 is refreshed. 

to on. embodiment the extraction agent operates in response to user commands to 
exBact *e correct .eve. of data ftom database 20. In one such embodiment, one could 
sy^em , 0 ,o respond .0 a command ^ch as "1 wan. these markets by these prod«,« for tlus 
pehod on the horizon, by channel" (or whatever your other dimensions are). Another 
c„mm«.d syntax might be "C3ive me a., my products and give me all my markets at the 
region level over a selected forecast horizon" (e.g.. current to cun:ent.p.us-12). 

to one such embodiment, refresh of the persistent cache is triggered by d.e current flag 
moving forward one month. For example, if the current month switches from Januao- to 
Febmary. a subset of data is read from database 20 and a combination of facts from dat^ase 
20 and calculate, fi^ts fom«d as a function of da« in database 20 axe stored in cache 16. 
Any da,aresid««inpe,sistentcaehel6a..hetimeof the refresh that isunsavedor 

uncommitted is simply overwritten. 

Tobuild*e work File, the systemlOmus. determine which dimensions are dnilable 

and a, which aggregation levels the nor^drillable dimensions will be forecast, to one 
embodiment.,hisinfo,mationiss.oredintheCa,.gory«4.1eoftheMetadaB. 

For drillable dimensions. daU is stored in the Wo* File « the lowest level defined m 
.he drill hierarchy for *at dimension. For non-driilab.e dimensions, data is stored only a. me 
levelofaggregadonspecifiedin the Metadataforfliat dimension. 

The override phase allows the user ,o interactively adjust the base staU^cal forecast 
dau DataisreadfromtheWorkFilegeneratedbytheexfractionphase. The user is allowed 
!5 «> adjust vah.es. lock ^ unlock values. driU up and down fte driUable dimens-on's 
hierarchy, as well as many other fimctions. 

once an adjuster has complced a s« of override changes, ftese changes are released 
.otoeworkfilesothatmeya^availableforfinalcomnnttothewarehouse. Therelease 
phase loads ti.e base data from the work file, loads an instance file which contiuns the saved 
30 set ofchanges. and writes the n«v values back to the woric file. 

The commit phase perfom,s the commit for the Override process. First. ., reads d.e 
WorkFile. Next, the da.a values of U,elowes.-level decomposed da« are read from fte 
warehouse. These values are used «, properly proportion tt.e data as i, is decomposed to ti« 
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lowest level in each dimension's hierarchy. Finally, the adjusted and decomposed data is 
written to the database. 

For example, suppose the data in the Work File is stored at the regional (REG) level 
of the Market dimension. Further suppose that the lowest level in the Market hierarchy is 
market (MKT). Store will read the REG level data from the work file, decompose the REG 
level data to the MKT level, and finally write the MKT level data to the database. 



The Database Structure 

In one embodiment, database 20 is implemented as a non-normalized star schema. A 
simplified version of a star schema 38 is shown generally in Fig. 3. In a star schema, facts are 
stored as data in fact tables. The fact tables are indexed by a multi-part key made up of the 
individual keys of each dimension. Similarly, dimension information is stored in dimension 
tables. In the embodiment shown in Fig. 4, DimTable 40 is a dimension table while the 
tables labeled "Warehouse Product", "Warehouse Market", "Warehouse Period" and 
"Warehouse Fact" are fact tables 42, 44, 46 and 48, respectively. 

Non-normalized star schemas are designed for very fast data aggregation and 
calculation. Such speed can be very advantageous in building the subset of data to be stored 
in cache 16. Such systems can, however, bog down considerably when required to perform 
incremental updates (e.g., as is the case during a forecasting session). That is where the use 
of cache 16 and instruction file 30 are instrumental in supporting such forecasting sessions. 

The downside of a star schema approach is that you end up with widespread 
replication of data across tables. At the same time, however, this replication of data give you 
the ability to get your keys from the data warehouse with very small queries that database 20 
certainly can handle quickly. So a star schema approach is very well optimized for queries 
that pull a large number of rows out of database 20. 

Standard OLAP reporting tools do not have to understand multiple levels within a 
product or market hierarchy of a multi-tiered data warehouse. This is a key difference 
between override engine 18 and a standard OLAP reporting tool. Override engine 18 must 
understand multiple levels at the same time because for the purposes of an adjustment, there 
may be interdependencies on the levels themselves for a reporting tool that essentially say, 
"O.K., here's data at this level," or "Here's data at the combination of these two levels." 
They're largely independent of each other and you can concentrate on one or the other at any 
given time. Because of that, override engine 1 8 does not run directly against the warehouse. 
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ovcnide engine ex«e« d.* p«*™. caloutations whe« necessary and s»re. 
the modified data to cache 16. 

I„ order to operate eorrecly, however, with daBbase 20. ovemde engme 18 and 
in.«c«onapp.ieaUonprocess34,n„«n™iers«.d*.a«uo»reofda.abase2^^^^^ 

embodiment, a. is shown in Figure 3. override engine . S a-K. ins.r„ctK>n apphc^on process 
34 extract the struchne of database 20 by reviewing the contents of Metadata table 36. 

such an approach permits the use of an unlimited and easily alterable n™nb« of 
dimensions.. nit. simples, form, metadatatable 36 incMesaperioddimens.ontable40a». 

a&«table46. to one embodiment, each dimension table includes a unique key. umque 

aescripUon field, a level colmnn and a hierarchy .eve, column. In addition, penod dmrensron 
«*,leinc.udesauni,uese,u«.ce„„mberco.umn.asequencewi,hinyearco,umnanda 

current pehod column. Fact table 46 includes keys which are identical in type and strucn^ 

to the keys listed in dimension table 40. 

For example. themet»lata.ablecanbeus«l.odriveadriUhierarchy,ha..ells 

database 20 that daysmakeup weeks and weeksmakeup ninths andmonthsmake up 

quarters, quarters make up halves, halvesmakeupyears. or that four luartersmdceupa 

year.so,hattheusercanjumpandskip.hi„gs.Samethingintheprod„cttae,a.chy. 

■ . lOnnHwaand the Metadata structure and use tne 
Override engine 1 8 and reportmg engine 19 understand tne iviewu 
data stored in the dimension tables to extract data ftom database 20. 

For example, a certain table may include a category of sales by total U.S^And may^ 
«iscateg»y.mam,facn.rer.brand.Andthenover in another table^^Us^^^^ 

approach is to have a separate fact table for each month's da.. That way 
when*e„«ttmon*-sdata arrive, it gets placed in.oanewfac.uble and s.oredmda.ab^ 

20. An advanurge of such an approach is *a. itcne of the fact tables gets lost ""O^P^^^ 
, youcanreloadthatmonth. to addition, as database 20 gets bigger and brgger. .lvalue to t^ 

company increases and so does the cost of maintaining the database. Relanona, da^bas.^^ 
scaJinto much larger da. sizes a»ia.ennKhmo.main.ainablethanacorrespondmgmu.u. 

dimensional database. 

I. should be understood that database 20 may be distributed across a number of 
,0 compute., to one embodiment, such as is shown in Fig. 5. compu,er sy^ern 50 includ«a 
servL 52 com,ec.cd .o a pro^ssor 58 and aprocessor 62 by a network 56. to a^ldrUon, serv. 
52 is connected to a plurality 1 through N of workstations 62. Processor 58 and processor 62 
Slore portions of database 20 in memories 60 and 64, respecOvely. 
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In one embodiment, as is shown in Fig. 5, server 52 maintains a separate instantiation 
54 of override engine 1 8, reporting tool 19 and cache 16 for each user perfomiing the 
forecasting function. In another embodiment (not shown), a single cache 16 is used across all 
users. In either case, changes to cache 16 are maintained in a separate instruction file 30 for 
each instantiation 52 and are only applied to the cache on receipt of a "release'* command. 

In one embodiment, processor 58 is a muhiple processor machine such as the Hewlett 
Packard HP9000 mnning an Oracle database application while processor 62 is a Tandem 
Himalaya 128. In one such embodiment, aggregated data is stored in the Oracle database on 
the HP machine while the lowest level data is stored in the Himalaya machine. The Metadata 
tells the program the appropriate processor to which a query should be addressed. For 
example, if data is stored by category by store in processor 62, override engine 18 can 
determine this be looking at the Metadata. If, on the other hand, the user is storing aggregate 
data in cache 16, it will determine fix)m the Metadata table that it should extract such data 
from the Oracle database on processor 58. 

In another embodiment, override engine 1 8 and reporting tool 19 run as a relatively 
thin windows client that essentially just provides GUI 32. All the analytics, all the storage, 
and all the real processing is done on UNIX server 52. 

liQcking Values in the Database 

In one embodiment, override engine includes the ability to lock values in database 20. 
As noted above, on a release, the changes at one level of the hierarchy are pushed down to 
each of the sublevels of the hierarchy. For example, if production is increased across the 
board by 10,000 imits, the 10,000 units are distributed proportionally across each of the 
entities at the lowest level of the hierarchy. 

In certain situations, such an approach does not make sense. For instance, if one of 
three manufacturing plants is operating at capacity, it makes little sense to distribute the 
increase proportionally to the plant operating at capacity. For such situations, override engine 
18 includes a locking mechanism which can be used to lock the output of any of the 
manufacturing plants to a certain value. Any increases are then distributed proportionally 
across the remaining, unlocked, manufacturing plants. 

In one embodiment, the adjustment report displays the values of the dimensions the 
user selects on a Template dialog screen. The only values that can be adjusted for the 
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fo^arefoundm^eun^adcdcdls. looked valu« arc shaded red and non-a4i»s.ab.. 

values are shaded yellow. ^ r.u^ 

Theuser can adjustavaluea. any level and*eeire«sripple«^ughmeres^of*e 

producrMerarchy. For cxan-pleafheor she adjus^aSKU value. *e values ofprc*^^^^^ 
higher levelschangco account for *e lower level adjus».eut or. if *euser«l,«s«ah.gher 

level value, the values for the low level products change as needed. 

Likewise, wh^ttheuserlocksavalue. product valuesbott. up and downthehterarchy 

n.y he affected. For cxantple. if the top level is locked at 100, all lower values must total 

100 

The dUgram shown in Fig. 6a is an example of decomposition or proportional fitting 
based on cunent forecast data. Tlte current adjustment dau> is retrieved at 
70 and. after an adjustment, is proportionally propagated through Brand level 72 to SKU level 

If.however..hemanuftcturcp,cduCvaluew»lockedatlcvel70andoneBrand 
product vatae was adjusted 6om 50 to 60, the adjustmem and its effects are shown m F.g^ 6b. 

AS noted above, in one embodiment hothreportmg toon9 and override »gme 18 
have the ability to limit the effect of commands such that the data re««ned ftom da^hase 20 
does not include the entire hierarchy for all dimensions. For instance, one may care about 
multiple levels of product dimension, but would Uke to Hmit the market adjustments to a 
par^cular level (e.g., make the market adjusunenu a. atotal US. ). In one ^ '^^ 
«,e user can select the level at which data is adjusted. For example, one would tell overr-de 
engine 18 to. "Make your product dimension drillablc, make your market dimensron non- 
driUab.e.makeyourperioddime„sio.non.iriUable." Adjusuncnts then must onlybednven 

down fl» product hierarchy for display; the override engine does not have to try to dnvett 

down the other hierarchies. As you add multiple drillahle dimensions, the amount of wotk 
««.hastoh,ppenwhe.youmakeadjustine„tsexpa.dsexponentiaUy.No.onlydoes,tgo 

down product, but i, has to go down product for every market, or every subtnark^. or ^ety 
submarine, of every submarket and you end up with a huge matrix of numbers to keep track 

DriUingallowstheusertodisplaythefectvaluesatdifferentproduClevels^bothup 
and down the pr«iuc. hierarchy. With drilling, one can see the aggregate values of a group of 
products, or can identify the specific values that went into the aggregate amount. 
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Drilling up works with any product where there are other products at a higher level. 
For example, one can drill up on a value if he or she is at SKU level 74 shown in Fig. 6a. 

Finally, a user can drill down with any product if he or she is at a level above the 
lowest level, SKU. For example, one can drill down if at manufacture level 70 in Fig. 6a. 
5 If, for instance, you want to adjust the amount of a particular soft drink sold in a 

particular city, you could either make that a drillable thing or a non-drillable dimension. You 
can do it in one of two ways. One is you can set your market dimension as drillable, and then 
poll the whole market hierarchy in order to drill down to the city and make the change. If, 
however, you did not need that flexibility in drilling and the associated performance impact 
10 of allocating the changes through that dimension, you could make the market dimension 

'non-drillable' and only see data at the city level. Then you can just grab whatever city you 
wanted and make your adjustments. But because you probably have the data stored at some 
level below city, once you do a "commit" it is going to drive the nimibers to the bottom of all 
the hierarchies. (In one embodiment, override engine drives the mraibers down to the bottom 
15 of all the hierarchies by modifying each number as a percent of contribution to the total.) 

For example, if given a market hierarchy including regions, cities within those regions 
and stores within those cities, you were going to modify units at the regional level, you only 
need to maintain data in cache 1 6 at that level. To do this, override engine 1 8 initiates a 
transfer of data at the atomic level, receives the data and pushes it up to the regional level. 
20 The aggregated regional data is then stored in cache 16 and commands from the user are 
applied against that aggregate data. Then, when you are ready to store the changed data, a 
"commit" command is executed and the modifications get drilled down to the atomic level in 
database 20. The resuh is that user commands get executed quickly at the regional level and 
get stored accurately at the atomic level. 
25 For example, one might determine what would happen if volume increased by 10% in 

a region such as the Great Lakes territory. The change would be made at the territory level 
and, when committed, it is distributed to the cities as a function of the percentage they 
contribute to the regional total. 

No matter the level where the adjustment is made, once the warehouse is updated, it is 
30 updated at the lowest atomic level and then in this case for reporting engine 19, then it can 

automatically pick up a report of it based on market hierarchy, period hierarchy, etc. All that 
is a trade-off between speed and flexibihty. You can go ultimately flexible if you've got 
enough iron behind the thing to drive it. And if you do not, then you can cut back. 
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Also from a security perspective, quite often if it is sales people that are using it for 
the adjustments, they do not want them to see other territories, for instance, other people's 
clients, other sales guys' clients. Otherwise they start playing games that you do not want 
them playing. It is a combination of those things. 
5 Override engine 18 understands all of reporting tool 19's filters and calculated fects; 

it also understands the hierarchies of system 1 0. Essentially everything in the Metadata that 
can be defmed, is read by reporting tool 19 and override engine 18 and they will use those 
definition. So you can create a calculated fact in reporting tool 19 and override engine 18 
will automatically see it and know how to use it. 
10 (An example of a calculated fact would be gross revenue. Gross revenue is calculated 

as units times price. You would not, therefore store gross revenue. Instead, you calculate it 
off of the two that you did store (price and units). Another calculated fact is forecast error. 
Facts calculated as a fimction of an actual fact and a forecasted fact are derived, not stored. 
Oveiride engine 18 automatically reads the definition of each calculated fact out of the 
15 Metadata and applies them on tiie fly to the data read &om database 20.) 

In one embodiment, forecasts are stored as separate databases within database 20. For 
instance, one could include a dimension labeled "scenario" into the warehouse. It can, 
therefore, be critical to have the ability to add extra dimensions to systems 10 and 50. 



Cache 

20 As was noted m connection with Figure 2, in one embodiment cache 16 is a persistent 

cache stored as a B-tree in a UNIX file system on processor 14. The B-tree allows you to 
essentially go with a much more compact data storage in sitiiations where the matrix of data 
is populated sparsely (such as in, for example, customer centric databases) yet, at the same 
time, a B-tree implementation does allow you to get reasonably fast access times. 

25 In one such embodiment, tiie persistent cache is implemented using standard Rogue 

Wave B-tree code. The performance of the Rogue Wave B-tree code can be enhanced by 
replacing the standard I/O stream (which is write character by character) witii an VO stream 
which writes data in large blocks of data. 

In addition, with a B-tree implementation, one can ti-ade off size for speed by 

30 increasing or decreasing the number of branches and subbranches in the tree. That is, tiie 

deeper you make the B-tree, the slower your access time but, at the same time, the smaller the 
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actual physical storage. In one embodiment, the structure of the B-tree is optraiized for size 
over speed. 

In more densely packed matrices of data, cache 16 can be implemented as a set of file- 
based arrays. Such arrays are very fast, requiring only simple calculations to get to the right 

5 locations. They can be, however, very inefficient space-wise. Typically, if one of the 

dimensions of the data warehouse is customer, and the number of potential customers is large, 
you are better off selecting a B-tree implementation for cache 1 6. 

Override engine 1 8 allows the user to grab shces of data firom a database that may be 
too big to comprehend in its entirety. Override engme 1 8 allows you to slice a portion of the 

10 database out, which sometimes in the industry they'll refer to as a datamark, automatically 
and intelligently through the user of metadata. The portion extracted can be dealt with either 
within or outside the database quickly and efficiently. Once the changes are in place, 
override engine 18 automatically and seamlessly pushes them back into database 20. The 
result is a database tool which eliminates the query traffic bottleneck of traditional approaches 

15 to relational database management systems. 

Although specific embodiments have been illustrated and described herein, it will be 
appreciated by those of ordinary skill in the art that any arrangement which is calculated to 
achieve the same purpose may be substituted for the specific embodiment shown. This 
application is intended to cover any adaptations or variations of the present invention. 

20 Therefore, it is intended that this invention be limited only by the claims and the equivalents 
thereof 
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What is claimed is: 

1 A data warehouse, comprising: ca*^^*^^- 

a processor connoted .o fl.e memory, v-herm *c processor mcludes: 

a cache; and 

an override engine; • 

..e^in a,e cache inches a subset of U« plurality of da» '^^ '^^^^ 
overHdee.gineex.rac.sda.at.m*e cache forviewtasbya.ser.mo.J«^^^ 
^e.o one or moreuser commands and savesmeusercommandstoaffle for .a«r 

application to the database. 

2 A data warehouse, comprising: * ^tri^^- 

■ i,^<.c»rlatahase having a plurality of data entnes, 
memory, wherein the memory mcludes a database navmg p 

a server com^ected to the memory, wherein the processor includes: 

a cache; and 

an override engine; and 
a client workstation connected to the server, 

wherein d.eserverope^a.esinr.spo„se..usercomma„as.os.o.asu.^^^^^^^ 
«ofda«.enMesi„.he cache. »herein«.ov.rrideengi««c.rac«da«froma.e^ 

rsel.heda.ae..c....e-e.^chen.«oH^n^^^^^^ 
modifiesa..datainresponsetooneormoreoftheusercommands,sav«ae 

: a file and opera.es in re^nse .o a commit command .0 modifies d.e da.ab.se based on d,e 

user commands stored in the file. 

3. An,e«»dofreportingdaU.romadaUwareho„se.*emed»dcomp,isingd»s«ps 
of: 

providing a server and a memory device; 

Lrins a da.abase in U.e memory device. w.«rein *e da.ab^ inciudes a piarah.y of 

data entries; 

exttacting a subset of data entries ftom die daabase; 
s.oring die subset of data entries on the server, 

modifyingthedataentriess«,r«lon*eserverinresponse.o«serc»mnands; 
reading dau. from d-e modified da« entries stored on the server, and 
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displaying the data to the user. 

4. A method of forecasting, comprising the steps of: 
providing a server and a memory device; 

storing a database in the memory device, wherein the database includes a plurality of 
data entries; 

extracting a subset of data entries from the database; 
storing the subset of data entries on the server; 

modifying the data entries stored on the server in response to user commands; 
storing the user conunands; 

reading data from the modified data entries stored on the server; 
displaying the data to the user; and 

modifying the database based on the stored user commands. 

5. A method of increasing the speed in which changes to a relational database are 
reflected back to the user, comprising the steps of: 

extracting a subset of data from the database, wherein the step of extracting includes 
the steps of: 

displaying a rqsresentation of the subset of data to the user; and 

storing the subset of data in a cache; 
receiving a data modification command; 
storing the data modification conunand in a file; and 

applying the data modification command against the subset of data stored in the 
cache, wherein the step of applying the data modification command includes the step of 
modifying the subset of data to reflect application of the data modification command. 

6. The method according to claim 5, wherein the step of applying the data modification 
command further includes the step of modifying the representation of the subset of data to 
reflect application of the command. 

7. The method according to claim 6, wherein the step of applying the command further 
includes the steps of: 

waiting for a "commit" conrunand; and 
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on receipt of the "conunit" coimnand, modifying data in the database to reflect 
application of the data modification command. 

8. The method according to claim 5, wherein the step of applying the command further 

includes the steps of: 

waiting for a "commit" command; and 

on receipt of the "commit" command, modifying data in the database to reflect 
application of the data modification command. 

9. A data stracture for updating a database, comprising: 

a plurality of user data-modification commands, wherein the plurality of user data- 
modification commands provide information for modifying information in a database (20). 

1 0. The data structure of claim 8, wherein: 

the plurality of user data-modification commands provide data-modification 
information for application to aggregated regional data stored in a cache (16), and 
modification information that gets drilled down to an atomic level in the database (20). 
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